Vector quantization of images

ABSTRACT

A method of compressing an image is described in which digital data signals in a 2-dimensional images are formed into an image data pyramid with a number of layers and each layer is processed to give a compressed encoding in an ordered list. The encoding with the largest quality gain factor is selected first and added to a compressed representation of the data array. This is repeated for the next largest gain factor and so on until a predetermined maximum is reached. Each layer of the image data pyramid corresponds to different frequency bands, the vector quantizations of these layers will only minimally interfere with one another. This allows a simple ordering of all possible gain contributions made by the compressed encodings, to the compressed representation. This in turn allows a straightforward selection of the compressed encodings having the largest quality gain factors, for compiling the compressed representation of the image.

The present invention relates to image compression, and in particular toimage compression techniques utilising vector quantization of imagedata.

BACKGROUND OF THE INVENTION

A two-dimensional image, which may for example be an individual image ormay be a frame of a video sequence, can conveniently be represented in acomputer system as a two-dimensional array of numbers where the numbersencode the brightness of a particular pixel to be displayed on a CRT orother display device. For data compression purposes it is oftenconvenient to represent the array of image data in a more coarselyquantized form. After compression the data is stored in less memoryspace, but this occurs at the cost of a loss of accuracy. Vectorquantization is a well-known enhancement which can be applied to imagecompression so as to obtain significant reductions in the number of bitsrequired to represent a given image in a computer system.

In vector quantization of a two-dimensional image a codebook ofreference patches (e.g. relatively small portions of an image taken fromone or more “library” images) is constructed. An image to be compressedis partitioned into a number of image patches and a matching (i.e.similar) reference patch is selected for each patch of the originalimage, from the codebook. The codebook index for each chosen referencepatch in the codebook is stored, together with addressing information(giving the positions of the image patches in the original image) toprovide a compressed representation of the image. The chosen codebookindex for a patch is often referred to as the “compressed encoding” ofthe patch. Providing that a copy of the codebook is available, anapproximation of the original image can be constructed by using thestored codebook indices to recover the required set of reference patchesand inserting these into an image frame using the respective storedimage patch position vectors.

WO97/16026 describes an improved vector quantization technique whichincreases the compression ratio which can be achieved in the compressedimage. This technique effectively decides what subdivision will becarried out for individual patches of the original image, when selectingmatching reference patches i.e. the size and position of the referencepatches from the codebook which will be used to replicate a given patchof the original image. This involves selecting from a plurality ofcompressed encodings that one of the compressed encodings which, whenuncompressed and added to a reference data array, gives the biggestimprovement therein relative to the original image data array. Themethod progressively adds to the reference data array the compressedencodings providing the greatest improvements to the reference array,until a predetermined maximum number of bits are present in thecompressed representation, or a desired quality level is achieved in thereference image data array.

One disadvantage of the technique described in WO97/16026 is that therecan be interaction between the improvements or “gains” of differentpatches at different scales, with the result that the image compressorhas to repeatedly re-evaluate the gains to be had from differentpatches. A combinatorial explosion of calculation times would berequired to try to calculate the best possible combinations of patches,and this is not practical to implement in the compression apparatus. Thetechnique can therefore result in non-optimal selections of patches.Moreover, the compressed image often has a “blocky” appearance due todiscontinuities at the edges of adjacent patches.

It is an object of the present invention to avoid or minimise one ofmore of the foregoing disadvantages.

SUMMARY OF THE INVENTION

According to one aspect of the present invention we provide a method ofcompressing an array of data entries in the form of digital electricalsignals, the array representing a two-dimensional image, the methodcomprising the steps of:

-   (a) forming an image data pyramid from the array of data entries,    said pyramid having a plurality of layers, a first and lowermost    said layer comprising said array of data entries, and each    subsequent higher layer comprising a data array representing the    image using less data, and having smaller dimensions, than the    previous layer;-   (b) processing each layer of the image data pyramid so as to obtain:    a set of compressed encodings for each layer, wherein each said    compressed encoding can be uncompressed to provide an approximation    of a block of image data derived from a respective layer of the    image data pyramid; and a respective quality gain factor associated    with each said compressed encoding;-   (c) creating an ordered list of the compressed encodings, ordered    sequentially according to the quality gain factors associated    therewith;-   (d) selecting the compressed encoding associated with the largest    quality gain factor and adding this compressed encoding to a    compressed representation of the data array;-   (e) selecting the compressed encoding associated with the next    largest quality gain factor and adding this compressed encoding to    the compressed representation of the original image data array; and-   (f) recursively repeating step (e) for the remaining compressed    encodings until the number of bits in the compressed representation    reaches a predetermined maximum.

The compressed representation to which the compressed encoding with thelargest quality gain is added the first time step (d) is carried out isan empty data file. As more and more compressed encodings are added tothis (initially empty) data file, the number of bits in the data file(and thus in the compressed representation) increases, until finally thepredetermined maximum number of bits is reached (this may, for example,be when an area of memory allocated for the data file is full).

Preferably, step (a) comprises forming a non-differential image datapyramid, having layers 0 to n, from the original image data array, andstep (b) comprises:

-   1) define a “current layer number” i and set i=n−1 where n is the    topmost (i.e. smallest) layer of the non-differential pyramid;-   2) divide layer n of the non-differential pyramid into a grid of    blocks which are, preferably, uniformly sized; determine a    compressed encoding for each said block in layer n of the    non-differential pyramid, wherein the compressed encoding can be    uncompressed to provide an approximation of the block; and determine    a quality gain factor associated with each said block in layer n of    the non-differential pyramid and its respective compressed encoding;-   4) replace each block in layer n of the non-differential pyramid    with its respective compressed encoding, in uncompressed form, so as    to form a current decompressed image at layer n;-   5) form an expanded version of the current decompressed image at    layer i+1 such that the expanded version has the same dimensions as    the pyramid layer i, and subtract this expanded version from the    pyramid layer i so as to form a differential pyramid layer i;-   6) divide the differential pyramid layer i into a grid of,    preferably uniformly sized, blocks; determine a compressed encoding    for each said block in the differential pyramid layer i, wherein the    compressed encoding can be uncompressed to provide an approximation    of the block; and determine a quality gain factor associated with    each said block in the differential pyramid layer i and its    respective compressed encoding;-   7) replace each block in the differential pyramid layer i with its    compressed encoding, so as to form a decompressed differential    pyramid layer i;-   8) add the decompressed differential pyramid layer i to the expanded    version of the current decompressed image at layer i+1, to form a    current decompressed image at layer i;-   9) subtract one from i (i.e. set i to i−1);-   10) recursively repeat steps (5) to (9) until the quality gain    factors for the lowermost layer (layer 0) of the differential    pyramid have been determined.

Alternatively, though not so preferred, step (a) may comprise forming anon-differential image data pyramid from the original image data array,and step (b) may comprise:

-   1) forming a differential image data pyramid from the    non-differential image data pyramid;-   2) dividing each layer of the differential image data pyramid into a    grid of blocks which are, preferably, uniformly sized, said blocks    preferably being the same size on each level of the pyramid;-   3) determining a compressed encoding for each block on each layer of    the differential pyramid, wherein the compressed encoding can be    uncompressed to provide an approximation of the block; and-   4) determining a quality gain factor associated with each block on    each layer of the differential pyramid and its respective compressed    encoding.

The compressed encoding for each said block preferably comprises acodebook index of an entry from a vector quantization codebook, whichentry most closely matches the said block. The compressed encoding mayadditionally include addressing information for the said blockspecifying the layer of the image data pyramid in which the block islocated, and the position vectors defining the position within the layerat which the block is located. Alternatively, and preferably, thecompressed encodings are listed in a predetermined order in thecompressed representation, such that the position in the original dataarray (i.e. the address) corresponding to a given encoding determinesthe position of that encoding in this list. The address for any givenencoding can thereby be determined by its position in the list ofcompressed encodings which makes up the compressed representation.

One advantage of the invention is that because the information at eachlayer of the image data pyramid corresponds to different frequencybands, the vector quantizations of these layers will only minimallyinterfere with one another. This allows a simple ordering of allpossible gain contributions made by the compressed encodings, to thecompressed representation. This in turn allows a straightforwardselection of the compressed encodings having the largest quality gainfactors, for compiling the compressed representation of the image.

A further advantage of the invention will be appreciated from thefollowing. In some prior art vector quantization techniques for imagecompression, each block of the original image data array is representedin the computer system as two components: a brightness shift and acodebook entry (i.e. compressed encoding). The brightness shift is addedto the codebook entry when reconstructing the image. The advantage ofthis is that the codebook can be made smaller if it does not have toinclude the same pattern at a whole range of brightnesses. This speedsup encoding and reduces the memory requirements of the decoder (i.e.decompressor) The disadvantage is that additional information(brightness shift) has to be sent to update each block of the image. Inthe method of the present invention, at all pyramid levels with theexception of the topmost and thus smallest layer, each block willinherit a background brightness level from the layer above it in thepyramid. The amount of information which must be used to represent eachblock is thus smaller than in the latter described prior art methods.This, in turn, allows higher compression ratios to be achieved with thepresent invention, as compared with such prior art methods.

A further benefit of the above-described first and preferred embodimentin which step (b) comprises the above-described steps (1) to (10), isthat in that embodiment each lower layer (i) of the pyramid iseffectively used to partially compensate for any artifacts introduced bymismatches between the codebook entries and the original blocks of datafrom the layer above (i+1). There are inevitably artifacts in anycodebook based compression scheme where the range of possible variety ofthe entries in the codebook is less than the range of possible varietyof the original data. One artifact that will occur when comparing anenlarged version of a decompressed image as compared to what one wouldget with simply shrinking and then re-expanding an image is that therewill be errors in mean brightness over groups of pixels. Suppose thateach layer is half the linear dimensions of the layer below, then apixel in layer corresponds to 4 pixels in layer i−1. Now if we simplyshrunk an image and re-expanded it, this would be equivalent, if we useda naive expansion algorithm, with simply replacing each block of 4pixels with their average. If the layer above has been vector quantizedor otherwise compressed, then the pixel that is expanded to 4 pixels mayno longer correspond exactly with the average of the 4 original pixels.Instead, it will have a ‘quantization’ error. Such quantization errorsare artifacts that, subject to availability of bits in the compresseddata channel, can be corrected in the vector quantized layer below.

Preferably, the invention also provides a method of decompressing thecompressed representation of the image data array obtained in the methodaccording to the above first aspect of the invention, so as to obtain areconstructed image data array, the decompression method comprising thesteps of:

-   (a) forming a reference image data pyramid, having the same number    of layers, 0 to n, and each layer having the same dimensions, as the    image data pyramid formed in step-   (a) of the above-described method, in which reference pyramid each    data entry is preferably set to mid-grey;-   (b) using the information in the compressed representation to    replace at least some portions of each layer of the reference data    pyramid;-   (c) defining the topmost (smallest) layer n of the reference data    pyramid to be the “current” layer-   (d) expanding the current layer of the reference data pyramid to    match (in dimensions) the layer immediately below the current layer    in the reference data pyramid, so as to form an expanded version of    said current layer, and adding said expanded version of the current    layer to said immediately below layer to form an updated version of    said immediately below layer;-   (e) defining the updated version of said immediately below layer to    be the current layer;-   (f) recursively repeating steps (d) and (e) until the updated    version of the lowermost layer in the reference data pyramid has    been obtained, which updated version of the lowermost layer is the    reconstructed image.

An additional advantage of the invention, which will be particularlyapparent from the above description relating to the decompressionprocess, is that in the decompressed version of the compressedrepresentation the block boundary artifacts (i.e. blocky appearance)that are often visible when using prior art compression/decompressiontechniques based solely on block encoding schemes (i.e. without usingthe image data pyramids according to the present invention) can beavoided as long as suitable interpolation, such as for example linearinterpolation, is used during the expansion of the pyramid layers.

According to another aspect of the invention we provide apparatus forcompressing an array of data entries in the form of digital electricalsignals, the apparatus comprising digital data processing apparatus,conveniently in the form of a computer system set up to run a programembodying the method according to the above-described first aspect ofthe invention.

According to a third aspect of the invention we provide a computerprogram product comprising a computer usable medium having computerreadable code means embodied in said medium for compressing an array ofdata entries in the form of digital electrical signals, said computerprogram product having computer readable code means for carrying out thedescribed steps of the method according to the first aspect of theinvention.

BRIEF DESCRIPTION THE DRAWINGS

Preferred embodiments of the invention will now be described, by way ofexample only and with reference to the accompanying drawings in which:

FIG. 1 illustrates the process of forming a differential image pyramidfrom a non-differential image pyramid;

FIG. 2 illustrates a linear interpolation technique;

FIG. 3 illustrates a differential image divided into a grid of squareblocks;

FIG. 4 illustrates an ordered list stored in a memory;

FIG. 5 illustrates schematically two adjacent layers of an imagepyramid; and

FIG. 6 is a schematic diagram illustrating an image compression systemaccording to one embodiment of the invention.

DETAILED DESCRIPTION

It will be understood that the images shown in FIG. 1 are digital imageswhich are each represented, in digital image processing, by atwo-dimensional array of data entries, each data entry representing thedigital electrical signal at one pixel in a two-dimensional pixel arraywhich displays the image. It will further be generally understood thatin the below-described image processing methods it is the arrays of dataentries representing each image which are operated on as described.

Uppermost in FIG. 1 is a non-differential image pyramid having threelayers 1,2,3. The lowermost layer 1 is an original image obtained, forexample, as a still frame from a video camera. This original image isused to form the two smaller image layers 2, 3, by first effectivelyshrinking the original image to form a half-scale image at the nextlayer 2 up in the pyramid, which next layer up is then itself shrunk toproduce an image at the uppermost (topmost) layer 3 of the pyramid. Thetopmost layer of the pyramid is quarter-scale in size, relative to theoriginal image (the lowermost layer 1). The larger images in the pyramidcontain information that is not present in the smaller ones. Theformation of such image pyramids is well known in the arts of computervision and image processing, and one technique is described, forexample, in “Structured Computer Vision, S Tanimoto & A Klinger,Academic Press 1980. Various techniques are nevertheless known forforming a non-differential image pyramid and any of these may be used toform the pyramid of FIG. 1.

Lowermost in FIG. 1 are three image layers 6,7,8 in a differential imagepyramid formed from the non-differential image pyramid layers 1,2,3. Thefull and half scale images in the differential pyramid are formed bysubtracting enlarged versions 4,5 of the half and quarter scale images(i.e. the two uppermost layers 2,3 are enlarged to full and half scalesize respectively) from the full and half scale images (the twolowermost layers 1,2) respectively in the non-differential imagepyramid, as illustrated in FIG. 1. When enlarging the image from onelayer to another, some appropriate smoothing interpolation is performed,for example a bi-linear or bi-cubic interpolation. Bi-linearinterpolation, where interstitial pixels are formed as the average ofthe pixels which surround them, is illustrated in FIG. 2 in which thelarge circles with labelled co-ordinates represent pixels in an image ofsize N and the small circles represent the pixels in an image of size(2N)−1 that are derived from them by bilinear interpolation when theimage in layer of size N is expanded to twice its size. Thus smallcircle a is the average of pixels (0,0) and (1,0), b is the average ofpixels (1,0) and (1,1) whilst e is the average of all of the largerpixels.

The topmost layer 8 in the differential pyramid is the same image as thetopmost layer of the non-differential pyramid. Again, the processes forforming differential pyramids are well known in the computer visionarts, and are described for example in the above-mentioned reference“Structured Computer Vision”. It should be noted that the differenceimages (i.e. the two lowermost layers 6,7) in the differential pyramidcontain information up to a certain frequency. This is the samefrequency for the two difference images and will in general be in therange between the Nyquist limit to just above half the Nyquist limit forthe scale at which they are represented. Since each image is half thesize of the previous one, this means that each contains informationcorresponding to roughly half the frequency of the previous one. (Theinformation will not be at exactly half the frequency since the processof expanding the smaller image before subtracting it from a larger onemay introduce artifacts in frequency space.)

FIRST EMBODIMENT OF THE COMPRESSION METHOD

We now propose an improved process for the vector quantization of videodata using the above-described pyramids. The method steps are asfollows:

-   1. Form a non-differential image pyramid from an original video    image (as described above);-   2. Convert this to a differential image pyramid (as described    above);-   3. Divide each layer of the differential pyramid into a grid of    uniform sized rectangular “patches” as shown in FIG. 3. The patches    are the same size on each layer of the pyramid. In the embodiment    illustrated in FIG. 3, a differential pyramid image layer is shown    divided into a grid of 7×7 square patches.-   4. For each patch in each layer determine the codebook entry, from a    vector quantization codebook, that best matches it;-   5. For each patch and its corresponding vector quantization codebook    entry determine the quality gain G as follows:

$G = {{\sum\limits_{i = 0}^{n}\;{\sum\limits_{j = 0}^{n}\;\left( P_{ij} \right)^{2}}} - {\sum\limits_{i = 0}^{n}\;{\sum\limits_{j = 0}^{n}\;\left( {P_{ij} - C_{ij}} \right)^{2}}}}$Where Pij stands for the pixels in the patch and Cij the correspondingpixels in the codebook entry. This can also be represented as: G=S−Rwhere

$S = {\sum\limits_{i = 0}^{n}\;{\sum\limits_{j = 0}^{n}\;\left( P_{ij} \right)^{2}}}$represents the total energy of the image within the patch, and

$R = {\sum\limits_{i = 0}^{n}\;{\sum\limits_{j = 0}^{n}\;\left( {P_{ij} - C_{ij}} \right)^{2}}}$represents the energy of the residual error within the patch after ithas been subjected to the compression and decompression process. Thus,the total gain G is G=S−R i.e. gain G is the source energy S minusresidual error. It will be appreciated that this gain measure G is basedupon the squared error values since we are assuming that pixels arerepresented as signed real numbers with 0 representing mid gray. Thecodebook indices for each patch and the associated gains G are stored inmemory locations reserved therefore in a single general purpose memorybank used for all the data which needs to be stored permanently ortemporarily.

-   6. Form an ordered list of the patches based upon their associated    gains, as illustrated in FIG. 4, the list being ordered such that    the patches are sequentially ordered according to ever decreasing    gain, with the patch having highest gain G being first in the list.    Techniques for forming such lists are well known. See, for example,    “Fundamental Algorithms”, D. Knuth, Addison Wesley publishing 1975.-   7. Add the codebook index for the patch having the highest gain G,    along with appropriate addressing information of the patch, into an    empty data file, thereby forming an initial compressed    representation of the original image. The addressing information    specifies the layer of the pyramid and the co-ordinates (i.e.    position co-ordinates) within the layer at which the patch occurs.    We refer to the chosen codebook index for the patch as the    “compressed encoding” of the patch;-   8. Continue to add the compressed encodings for the patches    successively lower in the ordered list (i.e. according to decreasing    gains G) to the compressed representation data file until the number    of bits in the compressed representation reaches some pre-determined    limit set by the compression ratio.

Methods for creating codebooks and searching them are well known in theart. The technique used in the present invention in this regard are thesame as those used and described in WO97/16026.

The Decompression Method

To decode the compressed representation we proceed as follows.

-   1. Form an image pyramid of the appropriate size (i.e. same number    of layers and size of layers as used in the compression method) all    of whose pixels on all of whose layers are set to mid grey (0 in the    present embodiment)-   2. Using the information in the compressed representation replace    each patch in the pyramid for which there is a compressed encoding    in the compressed representation with the corresponding code book    entry.-   3. Take the topmost layer of the pyramid, expand the image in size    until it is the same size as the layer below.-   4. Arithmetically add each pixel in the newly expanded topmost layer    to the corresponding pixel in the second from top layer, placing the    result of the addition in the corresponding pixel in the second from    top layer.-   5. Delete the topmost layer of the pyramid, so that the formerly    second from top layer becomes the top layer.-   6. Repeat steps 3 to 5 until there is only a single layer left in    the pyramid. This is then the reconstructed image.

SECOND AND PREFERRED EMBODIMENT OF COMPRESSION METHOD

In a second and preferred embodiment of the invention, the method forcompressing an original video image has the following steps:

-   1. Form a non-differential image pyramid with layers 0 to n, where n    is the topmost, and therefore smallest, layer of the pyramid.-   2. Allocate a buffer to hold the current representation of the    decompressed image, call this buffer B.-   3. Allocate a buffer to hold the current differential pyramid layer,    in uncompressed form (i.e. not ever compressed and then    decompressed), and call this buffer C.-   4. Allocate a buffer to hold the current decompressed “differential”    pyramid layer, call this buffer D.-   5. Allocate a register to hold the current layer number, i, call    this register i, and initialise this to i=n−1-   6. Copy into C layer n of the pyramid.-   7. Divide layer n in C into a grid of uniformly sized rectangular    “patches” and for each patch in C find the best matching codebook    entry and calculate the gain G associated therewith, using the same    formula for the gain G as in step (5) of the first compression    method, described above Then place the co-ordinates of this patch,    the codebook index for the best matching codebook entry, and the    associated gain G calculated therefor, in an empty data file the    contents of which will become an ordered list as in step (6) of the    first compression method as described above.-   8. Place into buffer D a decompressed image formed by replacing each    patch in C with the corresponding code book entry (i.e. the best    matching codebook entry found therefor in step 7).-   9. Copy contents of buffer D into buffer B.-   10. If the current layer number i in register i is less than zero go    to step 17.-   11. Expand the image in B until it is the same size as the image in    layer(i), using an appropriate interpolation technique as before,    and placing the expanded result back in buffer B.-   12. Subtract image B from layer(i) and store the result in buffer C.-   13. Divide the image in buffer C into a grid of uniformly sized    rectangular patches of the same size as in step 7 above; for each    patch in C find the best matching codebook entry, and calculate the    gain G associated therewith using the same formula as before; and    place the coordinates of this patch, the codebook index of the best    matching codebook entry therefor, and the associated gain G    therefor, in the ordered list started in step 7 above.-   14. Place into buffer D an image formed by replacing each patch in C    with the corresponding codebook entry.-   15. Add buffer D to buffer B.-   16. Subtract 1 from register i and go to step 10.-   17. Traverse the ordered list of codebook indices (ordered according    to decreasing gain G), placing the codebook indices having the    highest gains G into an empty data file, sequentially in order of    decreasing gain G (thereby creating a compressed representation of    the original image, of increasing bit size) until the data file (and    hence the compressed representation) is of a predetermined size    (i.e. number of bits).

In either of the above-described compression methods, the codebookindices for the selected patches of high gain G are placed in thecompressed representation data file in a predetermined order accordingto the respective addressing co-ordinates associated therewith. In thismanner it is not necessary to also store the addressing co-ordinates inthe compressed representation itself: the order of the codebook indices(these are the “compressed encodings” for the related patches) gives therequired addressing information for each compressed encoding.

The compressed representation of the original image is thus made up of aseries of “codewords”, each codeword consisting of a compressed encodingof a patch. This predetermined order in which the compressed encodingsare listed can be chosen from a number of possible orders. For example,the compressed encodings can be stored in “layers” i.e. all theencodings for one layer, followed by all the encodings for the nextlayer etc. Alternatively, the encodings may be stored recursively, asfollows:

Assume that we are considering a local region x on layer i of the imagepyramid such that region x can be compressed with a single vectorquantization codeword. We denote the vector quantization codeword for xas CodeWord(x). Suppose further that for all i≧1 on layer (i−1), that isto say, the layer immediately below layer i on the image pyramid, theregion of the image that corresponds to the region x on layer i, iscomposed of a rectangle of subregions A, B, C, D as shown in FIG. 5.

The codeword sequence generated for region x and all regions below it inthe pyramid we denote by CodeSeq(x). We define it as follows:

if x is on the lowest layer of the pyramid then codeseq (x)=CodeWord(x), otherwise

-   CodeSeq (x)=CodeWord (x) CodeSeq (A) CodeSeq (B) CodeSeq (C) Codeseq    (D)

A recursive order of this type is likely to be preferred where runlengthencoding is to be employed in the data handling. However, other suitableorders could alternatively be used, with or without runlength encodingmethods.

To decompress the compressed representation in order to obtain thereconstructed image, the same decompression method (steps 1 to 6) isused as already described above.

It will be appreciated that the advantage of the second and preferredcompression method is that each lower layer of the pyramid is able topartially compensate for artifacts (i.e. errors) introduced bymismatches between the chosen codebook entries and the associatedoriginal rectangles (patches) in the layer above, by performing theencode/decode process at the layer above and taking the result intoaccount when obtaining the compressed encodings for the layer below.

When the bandwidth for the compressed representation is restricted, thenvector quantization data for only a subset of all the possible patchesin the pyramid will be stored in the compressed representation e.g. asubset consisting of the chosen encodings having highest gain, asdescribed above. In this case it is still possible to use the positionof a codeword in the compressed representation to provide implicitaddressing information for that codeword. However, in this case theabove-described method for recursively storing the compressed encodingis modified so that CodeSeq(x) is now defined as:

if the compressed representation contains no codebook index for positionx and there are no codebook indices for positions below x in the pyramidthen output bit 0;

if the compressed representation contains no codebook index for positionx but it does contain information for one or more of the positions inthe pyramid below position x then output bits 10 followed by CodeSeq(A)CodeSeq(B) Codeseq(C) CodeSeq(D);

if the compressed representation contains a codebook index for positionx in the pyramid then output bits 11 followed by

CodeWord(x) CodeSeq(A) CodeSeq(B) Codeseq(C) CodeSeq(D).

Apparatus for carrying out the above-described first embodiment of thecompression method is illustrated schematically in FIG. 6 which showsthe original image I stored in a first buffer RAM 10. A microprocessor20 programmed to carry out all the image data processing operations hasread access to the first buffer RAM 10 and has read and write access toa series of image buffer RAMS 12 a, 12 b, 12 c, 12 d for storing thedifferential image pyramid layers respectively (FIG. 6 illustrates thecase where there are four layers in the differential image pyramid, thetop-most layer consisting of the original image I). The microprocessor20 also has write access to an output buffer 30 into which thecompressed representation of the original image is stored. In the secondand preferred embodiment of the compression method described above, theimage buffer RAMs 12 a,12 b,12 c,12 d are used for storing the layers ofthe non-differential image pyramid and the microprocessor 20 also hasread and write access to three temporary working buffers B, C, D (shownin broken line) which are the buffers allocated to hold the currentrepresentation of the decompressed image, the current differentialpyramid layer, and the current decompressed differential pyramid layer,respectively (see steps 2, 3 and 4 of the second and preferred method).

It will be appreciated that various modifications to the above-describedembodiments are possible without departing from the scope of theinvention. For example, pyramids having more than three layers could beused. In practice we envisage that approximately ten pyramid layers willbe provided.

It will also be appreciated that the feature of storing the compressedencodings in the compressed representation in a predetermined orderwhich is used to indicate the addressing information for each encodingis best suited for compression applications where full image frames arecontinually being compressed, sent, received and decompressed e.g. inhigh bandwidth video. However, where only a portion of a full frame ischanging/being updated (e.g. in low bandwidth video) it is likely to bepreferable to send particular addressing information for the compressedencodings relating to the changing/updated image portion. In this case,therefore, the compressed representation may also contain the addressingco-ordinates for some or all of the compressed encodings therein.

Furthermore, while in general the use of patches of uniform size andshape throughout the image pyramids is preferred in order to keep thesize (i.e. no. of entries) of the codebook to a minimum, patches ofdifferent size and/or shape could be used at different layers, or evenwithin a layer, of the pyramid (this would necessarily require a largercodebook). Moreover, it may be desirable to choose the shape of thepatches to match the shape of the original image e.g. rectangularpatches for a rectangular image. However, one could alternatively firstchange the shape of the original image (e.g. from rectangular to square)and then use a desired shape of patches (e.g. square).

Where the images are colour images it will be preferable to send all thecolour data (e.g. red, green and blue components) adjacently (i.e. inimmediate succession) for each pixel, rather than sending e.g. all redcomponents for all pixels, all green components for all pixels, all bluecomponents for all pixels. Thus, for colour images we assume that eachimage layer within the pyramid is made up of three planes holding thethree respective colour components of the image. These can be eitherred, green, blue or some other suitable tristimulus encoding of colourdata. We define CodeWord(X) to be given by the sequence CodeWord(Xr)CodeWord(Xg) CodeWord(Xb)

Where Xc with c ranging over the set {r, g, b} stands for one of thedifferent colour planes that corresponds to region X of the layer.

1. A method of compressing an array of data entries, the arrayrepresenting a two-dimensional image, the method comprising the stepsof: (a) forming an image data pyramid from the array of data entries,said pyramid having a plurality of layers, a first and lowermost layerof said layers comprising said array of data entries, and eachsubsequent higher layer comprising a data array representing the imageusing less data, and having smaller dimensions, than the previous layer;(b) processing each layer of the image data pyramid so as to obtain: aset of compressed encodings for each layer, wherein each said compressedencoding can be uncompressed to provide an approximation of a block ofimage data derived from a respective layer of the image data pyramid;and a respective quality gain factor associated with each saidcompressed encoding; (c) creating an ordered list of the compressedencodings, ordered sequentially according to the quality gain factorsassociated therewith; (d) selecting the compressed encoding associatedwith the largest quality gain factor and adding this compressed encodingto a compressed representation of the data array; (e) selecting thecompressed encoding associated with the next largest quality gain factorand adding this compressed encoding to the compressed representation ofthe original image data array; and (f) recursively repeating step (e)for the remaining compressed encodings until the number of bits in thecompressed representation reaches a predetermined maximum.
 2. A methodas claimed in claim 1 wherein the compressed representation to which thecompressed encoding with the largest quality gain is added the firststep (d) is carried out is an empty date file.
 3. A method as claimed inclaim 1 wherein step (a) comprises forming a non-differential image datapyramid, having layers 0 to n, from the original image data array, andstep (b) comprises: 1) defing a current layer number i and settingi=n−1, where n is the topmost layer of the non-differential pyramid; 2)dividing layer n of the non-differential pyramid into a grid ofuniformly sized blocks determining a compressed encoding for each saidblock in layer n of the non-differential pyramid, wherein the compressedencoding can be uncompressed to provide an approximation of the block;and determining a quality gain factor associated with each said block inlayer a of the non-differential pyramid and its respective compressedencoding; 3) replacing each block in layer n of the non-differentialpyramid with its respective compressed encoding, in uncompressed form,so as to form a current decompressed image at layer n; 4) forming anexpanded version of the current decompressed image at layer i+1 suchthat the expanded version has the same dimensions as the pyramid layeri, and subtract this expanded version from the pyramid layer i so as toform a differential pyramid layer i; 5) dividing the differentialpyramid layer i into a grid of uniformly sized blocks; determining acompressed encoding for each said block in the differential pyramidlayer i, wherein the compressed encoding can be uncompressed to providean approximation of the block; and determining a quality gain factorassociated with each said block in the differential pyramid layer i andits respective compressed encoding; 6) replacing each block in thedifferential pyramid layer i with its compressed encoding, so as to forma decompressed differential pyramid layer i; 7) adding the decompresseddifferential pyramid layer into the expanded version of the currentdecompressed image at layer i+1, to form a current decompressed image alayer i; 8) subtracting one from i; 9) recursively repeating steps (5)to (9) until the quality gain factors for to lowermost layer of todifferential pyramid have been determined.
 4. A method as claimed inclaim 1 wherein step (a) comprises forming a non-differential image datapyramid from the original image data array, and step (b) comprises; 1)forming a differential image data pyramid from the non-differentialimage data pyramid; 2) dividing each layer of the differential imagedata pyramid into a grid of uniformly sized blocks, said blocks beingthe same size on each level of the pyramid; 3) determining a compressedencoding for each block on each layer of the differential pyramid,wherein the compressed encoding can be uncompressed to provide anapproximation of the block; and 4) determining a quality gain factorassociated with each block on each layer of the differential pyramid andits respective compressed encoding.
 5. A method as claimed in claim 1wherein the compressed encoding for each block comprises a codebookindex of an entry from a vector quantization codebook, which entry mostclosely matches the said block.
 6. A method as claimed in claim 1wherein the compressed encoding additionally includes addressinginformation forte said block specifying the layer of the image datapyramid in which the block is located, and the position vectors definingthe position within the layer at which the block is located.
 7. A methodas claimed in claim 1 wherein the compressed encodings are listed in apredetermined order in the compressed representation, such that theposition in the original data array corresponding to a given encodingdetermines the position of that encoding in this list.
 8. A method ofdecompressing the compressed representation of the image data arrayobtained in the method according to claim 1, so as to obtain areconstructed image data array, the decompression method comprising thesteps of: (a) forming a reference image data pyramid, having the samenumber of layers, 0 to n, and each layer having the same dimensions, asthe image data pyramid formed in step (a) of claim 1, in which referencepyramid each data entry is set to mid-grey; (b) using the information inthe compressed representation to replace at least some portions of eachlayer of the reference data pyramid; (c) defining the topmost layer n ofthe reference data pyramid to be the current layer; (d) expanding thecurrent layer of the reference data pyramid to match the dimensions ofthe layer immediately below the current layer in the reference datapyramid, so as to form an expanded version of said current layer, andadding said expanded version of the current layer to said immediatelybelow layer to form an updated version of said immediately below layer;(e) defining the updated version of said immediately below layer to bethe current layer; (f) recursively repeating steps (d) and (e) until theupdated version of the lowermost layer in the reference data pyramid hasbeen obtained, which updated version of the lowermost layer is thereconstructed image.
 9. Apparatus for compressing an array of dataentries, the apparatus comprising digital data processing apparatus, inthe form of a computer system set up to run a program embodying themethod according to claim
 1. 10. A computer program product comprising acomputer usable medium having computer readable code means embodied insaid medium for compressing an array of data entries, said computerprogram product having computer readable code means for carrying out thedescribed steps of the method according to claim 1.